feat: add event metadata as json to publish error logs #188
Conversation
| @@ -247,6 +247,7 @@ def test_full_event_data_present_in_key_extraction_error(self, mock_logger, *arg | |||
| assert f"sourcehost='{metadata.sourcehost}'" in error_string | |||
| assert "event_data_as_json='{\"test_data\": {\"course_id\": \"id\", \"sub_name\": \"name\"}}'"\ | |||
| in error_string | |||
| assert f"event_metadata_as_json='{metadata.to_json()}'" in error_string | |||
There was a problem hiding this comment.
[observation] It seems some of the metadata is now duplicated. I guess that's not a big deal if this is just for errors.
There was a problem hiding this comment.
It is indeed duplicated. I thought we'd be able to use the columns from Splunk to make a metadata object, but the problem is there's a collision in the "source" field, which Splunk uses to store other data.
There was a problem hiding this comment.
I suppose on further reflection we could have just changed how the source shows up in the string representation but that would mean making a change specifically for Splunk!reasons in openedx-events, whereas the json representation seems like a more general solution. I don't think it's too much of a load on the logs to add this, especially because we only log on failure.
Add metadata in json form to the error logs when events fail to publish. This will be useful for republishing the events later.
Issue:
edx/edx-arch-experiments#354
Merge checklist:
Check off if complete or not applicable: